Visual Mapping of Aggregate Causal Frameworks for Constructs, Relationships, and Meta-Analyses

ABSTRACT

A method and system for extracting from the scientific, technical and academic literature constructs and causal relationships between such constructs, searching said literature and visualizing its contents in the form of aggregated maps centered around constructs and relationships of interest, the maps including construct maps, relationship maps, model maps, and meta-analysis maps.

CROSS-REFERENCE TO RELATED APPLICATIONS

The instant Application is a continuation of U.S. application Ser. No.15/595,942, filed on May 15, 2017 and currently pending, which is acontinuation of U.S. Provisional Application No. 62/336,578, filed onMay 14, 2016 and now expired.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

This invention was made with Government support under a National ScienceFoundation Grant, Award No. 1622260. The Government has certain rightsin this invention.

BACKGROUND Technical Field

This disclosure relates generally to the field of generating, searchingand visualizing knowledge-based models. More particularly, thedisclosure relates to parsing language in scientific, technical andacademic literature to extract asserted construct relationships, respondto queries thereof and visually represent query results.

Background Art Description

In many situations, academics and other researchers want to examine abody of scientific, technical and/or academic literature to findarticles most relevant to the subject matter of interest to them.

One commonly used academic search tool is Google Scholar. Google Scholarindexes research papers and other academic literature by paper-levelmetadata such as article and journal titles, author, and abstract. Italso indexes literature by keywords (words or phrases). These searchtools typically select and rank literature by predicted relevance basedon user-specified search criteria constructed using keywords, and insome cases based on search constraints indicated by boolean operatorsand/or proximity operators between such keywords.

A researcher interested reviewing, for example, literature discussingthe effects of “product variety” on consumers, may use Google Search.

FIG. 1 illustrates one embodiment of a screen capture of a first page ofa Google Scholar results in response to using “product variety” and“consumer” together in the search input for Google Scholar. As a result,documents are ranked by predicted relevance based on this search string.The algorithms may be complex, but the predicted relevance ranking maybe based on factors such as whether each document includes all thesearch terms, the location and frequency of each search terms in thedocument, and perhaps the relative proximity of the search terms. Thismethod works to some extent in producing search results that may happento be associated with the subject matter intended to be represented bythese search terms.

For example, the first Google Scholar result shown in FIG. 1 is a paperentitled “Consumer Surplus in the Digital Economy: Estimating the Valueof Increased Product Variety at Online Booksellers.” This articleappears to be related to the effects of product variety on consumers inthe context of online booksellers.

FIG. 2 illustrates one embodiment of a screen capture of a first page ofa Google Scholar results in response to a more specific query than theone above: “the effect of ‘product variety’ on consumers”. This morespecific query may have the unintended consequence of excluding relevantresults that use slight semantic deviations from the expression of therelationship between search terms.

In this example, the sixth Google Scholar result shown in FIG. 2 is apaper entitled “Research Commentary—Long Tails vs. Superstars: TheEffect of Information Technology on Product Variety and SalesConcentration Patterns.” This paper appears to examine the effects ofother constructs on product variety rather than the effect of productvariety on other constructs. This researcher may be interested in onlythe latter relationships with product variety.

When search results are under inclusive, a researcher may get anincomplete picture of the existing literature, potentially causing theresearcher to unintentionally duplicate previously established findingsin the literature or draw incorrect conclusions about the body ofliterature. When search results are over inclusive, a researcher mayspend a significant amount of time reading through literature to excludethose that are not relevant.

Additionally, consumers of scientific research may want to get aneasy-to-process overview of a body of scientific literature, preferablyin a graphical format that highlights the building blocks of theunderlying research, which are typically the constructs examined in thatresearch and the relationships examined between them.

What is needed are systems and methods to better characterize, classify,and search scientific, technical and academic literature, and to bettervisualize the variables and causal relationships investigated by thisliterature.

DESCRIPTION OF THE DRAWINGS

FIG. 1 is a screen capture of a first page of Google Scholar results forthe search term ‘product variety’+consumers

FIG. 2 is a screen capture of a first page of Google Scholar results forthe search term effect+of+‘product variety’+on+consumers

FIG. 3 illustrates one embodiment of a system for searching andretrieving constructs and causal relationships between constructs.

FIG. 4 illustrates one embodiment of a schematic representation of aconstruct map.

FIG. 5 shows one embodiment of a schematic representation of arelationship map.

FIG. 6 illustrates one embodiment of a schematic representation of amodel map.

FIG. 7 illustrates one embodiment of a user interface including a visualrepresentation of a construct map.

FIG. 8 illustrates one embodiment of a user interface including a visualrepresentation of a relationship map.

FIG. 9 shows one embodiment of a process for performing a user query.

FIG. 10 shows one embodiment of a process for extracting constructs andcausal relationships.

FIG. 11 illustrates one embodiment of a diagrammatic representation ofan embodiment of a machine 900, within which a set of instructions forcausing the machine to perform one or more of the methodologiesdiscussed herein may be executed.

DETAILED DESCRIPTION

This disclosure relates generally to the field of generating, searchingand visualizing knowledge-based models. Various examples of embodimentswill be described below with reference to the drawings. The followingexemplary embodiments are illustrative and are not to be construed aslimiting.

Researchers typically express their investigations in terms ofhypotheses and other construct relationships. The subject matter of suchinvestigations into hypotheses and construct relationships are common inthe social and behavioral sciences, but are also used extensively inother scientific and technical disciplines. These investigations may bereported in academic journals and other magazines, books, reports,documents, and other references. These information sources will becollectively referred to herein as papers or literature, but it will beunderstood that these papers are often embodied in one or more physicalforms. For the purposes described herein, physical papers are preferablyconverted to electronic documents for computer processing.

A construct can be defined as construct or category of interest that isstudied and empirically tested in a discipline, particularly in a socialscience discipline. (e.g., “product variety” or “consumer attitudes”would be considered constructs that are often researched in consumerpsychology). A “construct” is often synonymous with “variable,” thoughvariables can also be thought of as the representation of a constructwithin a specific study.

A construct relationship may be a hypothesis tested according to a validexperimental design which may allow for a scientifically-based inferenceto be drawn as to causality, even if the conclusion based on the resultsof that testing does not support a finding of causality. However, aconstruct relationship as used herein may also include lessscientifically rigorous investigations into relationships betweenconstructs. In some embodiments, a construct relationship may includeinvestigations to determine correlations and other associations betweenconstructs where causation is not established, or even evaluated.

When researchers search the academic literature, they are generallyinterested in finding papers that investigate similar or relatedhypotheses or construct relationships. These constructs are oftenrepresented as variables within the context of a particular empiricalinvestigation. There may be multiple investigations (also referred to asstudies) in a paper.

Some hypotheses are descriptions of causal relationships between two ormore constructs. Hypotheses typically indicate which of one or moreconstruct(s) predict or influence one or more other constructs (i.e.,the direction of the causal effect) and how changes in one or morepredictors (often represented based on independent variables) arerelated to changes in one or more outcomes (often represented asdependent variables).

In some hypotheses, there are constructs that are moderator constructs(often represented with moderator variables) that affect the sign and/orthe strength of the relationship between at least one independentconstruct and at least one dependent construct. A moderator construct issaid to interact with an independent construct when the addition of themoderator construct changes the effect of the independent construct onthe dependent construct.

In some hypotheses, there are constructs that are mediator constructs(often represented with mediator variables) that explain therelationship between at least one independent construct and at least onedependent construct, meaning that they are the mechanism that makes therelationship happen. When a mediator causes complete mediation it meansthat, in the absence of the mediator's effect, there would be no causalrelationship between an independent construct and an otherwise dependentconstruct. For example endorphins are a mediator construct in therelationship between chocolate and mood; chocolate (independentconstruct) increases a person's mood (dependent construct) becauseeating chocolate releases endorphins (mediator construct).

Mediator constructs can also interact with a moderator construct, suchthat a mediator's effect on the relationship between an independent anddependent construct is affected in sign and/or strength by the presenceof a moderator construct.

Multiple hypotheses can be integrated into a causal framework (sometimesreferred to as a conceptual framework or simply a framework). Thiscausal framework is a network of causally interlinked constructs thattogether provide a quick overview of the causal relationships examinedwith regard to one or more constructs, or of all the moderating andmediating relationships examined within the context of the causalrelationship between two or more constructs. This causal framework maybe used to organize a literature review so that various papers areassociated with particular constructs or particular relationshipsbetween constructs. The causal framework may be used to generate avisual representation that conveys a summary state of the literaturewith respect to the constructs that are incorporated into the causalframework.

The causal framework may be also be used to generate a visualrepresentation of the hypotheses that were tested in a paper or in astudy within a paper, which together form the causal model for thatpaper or study.

Natural Language Processing & Machine Learning

A system for searching and retrieving constructs and causalrelationships between constructs (illustrated in FIG. 3) and a processfor extracting constructs and causal relationships (illustrated in FIG.10) are discussed below.

Step 1: Study Identification

Identify Studies in a Paper:

This will be based on the study heading (which typically contains adescriptive word such as “study” or “experiment” and an identifyingnumber, e.g., “study 1”) along with other identifying data such as theline breaks between the study title and other paper sections

The underlying logic is that each study will have a unique heading, thatthe studies in a paper are arranged back-to-back in a paper (such that aparticular study ends right before the heading of the next study starts,and that the last study ends right before the general discussion sectionof the paper, which is typically marked by a heading such as “generaldiscussion,” “conclusion,” or “summary”)

Step 2: Identification of the Study Sections & Sentences

The following sections of a study may be identified:

All the section headings/subheadings (whereby a sectionheading/subheading is a piece of text that is typically not afully-formed grammatical sentence, and which is separated from thesection it refers to either by one or more line breaks, or by a periodfollowed by a full sentence; differentiating between a section headingand sub-heading is not really necessary, we're simply using both forsemantic comprehensiveness)

All the tables and figures, along with their corresponding headings

All the hypotheses present in a study, whereby a hypothesis has the formof a block of text separated by line breaks from the rest of the text,and which typically starts off with either “Hypothesis” or “H” followedby a number, and is followed by a colon.

The study gets parsed into individual sentences

Step 3: Identification of Variable Spans and Key Structures (at theSentence Level)

The model would parse each sentence at a time (whereby section headingscould also count as sentences) to identify variable spans and keystructures.

Using a rule-based approach (where the model is being provided with aset of pre-defined rules that specify the key structures), a puremachine-learning approach (where the model is trained on a training setand learns to recognize key structures), or a combination thereof, foreach sentence the model would identify whether the sentence contains anyvariables and/or key structures.

Key structures in a study are textual structures that help identify thecausal role of one or more causal variables in a study, and which can bepart of a sentence or a heading

Key structures always appear in combination with one or more variables,but variables do not always appear in combination with a key structure

Heading-based key structures declare the types of variables that will bediscussed in the section to which the heading pertains

If a section heading contains phrases like “independent variables” or“dependent variables,” the system will infer that all the variables thatare discussed in that section will be of the causal role specified bythe section heading

Sentence-based key structures contain one or more variables along withspecific keywords or expressions that declare or signal the causal roleof one or more variables, or the causal relationship between two or morevariables, e.g., Declarations of causal roles: “the independentvariables were variables A and B” or “variable A served as independentvariable”

Keyword-based signaling: e.g., a sentence that associates a variablewith a “stimulus,” with being “manipulated,” having certain“conditions,” or being a certain type of “factor” suggest that thatvariable is an independent variable

Expression-based signaling: e.g., a sentence that contains expressionsof the form “2 (variable A: level 1 vs. level 2)×2 (variable B: level 1vs. level 2) design” suggest that variables A and B are independentvariables; “an ANCOVA on variable C” suggests that variable C is adependent variable; “the effect/impact of variable A on variable C”suggest that variable A is an independent variable and variable C is adependent variable (within the same causal relationship); “anexplanation of an effect through variable D” suggests that variable D isa mediator

The above is simply one embodiment of the approach we can take. Themodel can also be configured to ingest not a sentence at a time, but aspecific number of words at a time, a paragraph at a time, a study at atime, or the entire document at a time. Also, in order to identify thesekey structures, several machine learning approaches can be used,including Conditional Random Fields, Word/phrase/paragraph vectors,Logistic model trees, Support vector machine, Shallow and Deep/recurrentartificial neural networks, or combinations thereof. Moreover, machinelearning model used does not need to rely on a pattern recognitionapproach in order to identify the key structures discussed above; anyapproach would work as long as the model takes as input words (at thelevel of a word sequence of a particular length, a sentence, aparagraph, a study, or a document) and provides as output variables andtheir causal roles.

Step 3-1: Sentence Categorization

Each sentence may be categorized as:

Not containing any variables (e.g., “This study used a similar design asStudy 2” or “The results were replicated”); in this case the sentencegets a ‘score’ of 0

Containing variables, but no key structure (e.g., “We measured variableA using measure X” or “The mean for variable A was Y”); in this case thesentence gets a ‘score’ of 1

Containing variables, along with key structures (e.g., “The impact ofvariable A on variable B was non-significant” or “Variable A wasmanipulated the following way”); in this case the sentence gets a‘score’ of 2

For all the sentences with a score of 2, the system would try todetermine, based on the template that each key structure in the sentencebelongs to, what the causal role is for each variable that is part of akey structure

Step 3-2: Key Structure Categorization

Some key structures declare a variable's causal role unambiguously(perfectly diagnostic structures), others do not (imperfectly diagnosticstructures)

The perfectly diagnostic ones would be assigned a maximum confidencescore (e.g., 100%), meaning that when a variable is encountered in thecontext of that structure, the variable's causal role can be determinedwith full confidence; no more information is needed from the study todetermine that variable's causal role

The imperfectly diagnostic ones would be assigned a confidence lowerthan 100%, meaning that when a variable is encountered in the context ofthat structure, the variable's causal role can be determined with high,but not full confidence; more corroborating information is needed fromthe study to determine that variable's causal role

Examples of perfectly diagnostic structures: “the independent variableswere variables A and B” or “variable A served as independent variable”or “we manipulated variable A” or “variable A had two levels: high andlow” (which suggests that variable A is an independent variable) or “thestudy used a 2 (variable A: level 1 vs. level 2)×2 (variable B: level 1vs. level 2) design”—confidence score of 100% for variable A being anindependent variable

Examples of mixed diagnostic structures: “the impact of variable A onvariable B was significant” (which suggests that variable A is mostlikely an independent variable, though it can also be acovariate/control variable, and variable B is most likely a dependentvariable, though it can also be a covariate; if variable A is acovariate/control variable, then variable B is necessarily a dependentvariable, and if variable B is a covariate/control variable, thanvariable A is an independent variable);

Compare this to perfectly diagnostic structure “the impact of variable Aon variable B was mediated by variable D” (which suggests that variableA is definitely an independent variable, variable B is definitely adependent variable, and variable C is definitely a mediator, because amediator always mediates the effect of an independent variable on adependent one)

Categorization by Exhaustiveness:

Certain perfectly diagnostic key structures not only express avariable's causal role unambiguously, but they are also exhaustive inthe sense that they specify all the variable of a particular causal rolethat appear in a study

This allows the system to infer that any other, unique variablesencountered outside of the context of that key structure willnecessarily have a causal role other than the one specified by the keystructure

Examples include the following:

If a section heading contains a phrase like “independent variables,” thesystem infers that that section contains only independent variables, andthat there are no other independent variables in the study; same for alldependent variables, mediators, and covariates

If a sentence contains a key structure of the type “the independentvariables in this study were A, B, and C,” the system infers that thatstructure contains only independent variables, and that there are noother independent variables in the study; same for all dependentvariables, mediators, and covariates

If a sentence contains a key structure of the type “2 (variable A: level1 vs. level 2)×2 (variable B: level 1 vs. level 2),” the system infersthat that structure contains all the independent variables in the study;this does not work for other types of variables, it's an expression thatis specific to independent variables only

Step 4: Causal Role Determination based on Key Structure Diagnosticityand Exhaustiveness

The system combines the variables identified in a study with the keystructures in which those variables appear to determine the causal rolefor each variable

The variables paired with a perfectly diagnostic key structure getassigned a confidence score of 100%, meaning that when a variable isencountered in a context of that structure, its causal role can bedetermined with full confidence; no more information is needed from thestudy to determine that variable's causal role

If one or more variables are paired with a perfectly diagnostic keystructure that is also exhaustive, then the system infers with 100%confidence that the other, distinct variables encountered outside ofthat perfectly diagnostic key structure have a causal role other thanthe one specified by that key structure

The variables paired with an imperfectly diagnostic key structure getassigned a confidence lower than 100%, meaning that when a variable isencountered in a context of that structure, its causal role can bedetermined with high, but not full confidence; more corroboratinginformation is needed from the study to determine that variable's causalrole

For these variables, the system tries to identify at least one sentencein which that variable is paired with a perfectly diagnostic keystructure; if such a structure cannot be identified, the system makes adetermination of the variable's causal role by going through all the keystructures paired with that variable and computing an overallprobability statistic for that variable's causal role

The system can learn over time that in certain key structures, variableshave a certain probability of having a particular causal role, e.g., inthe structure “the impact of variable A was significant,” variable A isan independent variable in 80% of the cases, and a covariate in theremaining 20%; hence, it will assign a confidence score of 80% forvariable A being an independent variable

Other Rules for Causal Role Determination:

Once a variable's causal role is determined with 100% confidence in astudy once, it does not change again, with the following exceptions:

studies that are poorly written (for example a variable may be declaredas a covariate at some point, and then as a moderator (i.e., independentvariable) later on)

mediators are sometimes expressed as DV's in the beginning of a study,and declared as mediators only later on; this issue can be solved bytreating each DV as being a potential mediator, hence allowing for adual role there

Unless there is clearly contradictory information about a variable'scausal role later on in a study (see above), each variable and itscausal role get committed to the database

Each variable gets assigned a unique ID that specifies its causal rolealong with a number (i.e., IV1, DV2, MD1, etc.)

Determine whether a study contains causal or correlational relationships(if, using the steps above, no variables can be identified that can becategorized as an independent or dependent variables, it can be inferredthat it is not a correlation/causation study and can be skipped)

Step 5: Variable Categorization by Concept

Each variable in a study gets assigned to a concept retrieved from aninternal or external ontology/thesaurus (whereby a concept is astandardized term that is stored in the ontology/thesaurus along withall its semantic variations (synonyms) and potentially other dimensionssuch as measures or related concepts)

The assignment happens by checking the variable name against theontology and looking for a match

If no match is found, the system marks the variable as “new” in theontology, and this variable can get flagged for review by a human

This approach is useful because in some cases, the system may identifytwo variables, for which it not have enough information to identifywhether they are distinct variables or not (e.g., one sentence says “Weare examining, among others, the impact of product variety on choice”,the other says “We are using the same measures as in Study 1, except forthe measure for product selection.” Do “choice” and “product selection”refer to the same variable? Classifying each against the existingconcepts in an ontology helps answer that question)

A similar approach would also be needed for identifying whethervariables that have different names in different studies should betreated as the same variable (e.g., when a variable may be called“product variety” in one study and “product assortment” in another)

Variables assigned to the same concept in the ontology are consideredthe same variable, either within a study or across studies

All variables successfully assigned to a construct get stored in theconstruct database, along with the variable's correspondence with theconstruct, and the study and paper it belonged to.

General Assumptions:

Unless otherwise specified, all the independent variables affect all thedependent variables in a study

Unless otherwise specified, all the independent variables interact witheach other in a study

Every mediator is also a dependent variable; every dependent variable ishence also a potential mediator

A variable (or co-occurrence of variables) that appears in all thestudies in a paper is considered the main variable (or co-occurrence ofvariables) in that paper

Construct Database

A construct database contains an index of constructs that were extractedfrom a set of research papers or other documents. Each record in thedatabase may be associated with a particular study. Some or all of thefollowing information may be stored in the construct database:

A standardized construct name for each construct used in the study. Thestandardized name may be different the actual name used for theconstruct in the study.

When extracting the constructs from each document, the indexing systemrecognizes variations in the same or similar construct used across thestudies in a single paper and across papers. For example, the construct“product variety” may be alternatively expressed as “assortment,”“choice set,” “set size,” and “number of options.” In order to mapsimilar or identical causal relationships to each other, thesealternative descriptions or synonyms of the same construct should berecognized as such, assigned to a standardized construct name in thedatabase, and stored as an “alternative name” for that construct. In apreferred embodiment, the standardized construct name is assigned aunique construct identifier, while each alternative construct name mayalso have its unique identifier.

The standardized construct name may be associated with a broad range ofconstruct names used in various studies to represent the same or similarconstruct. These associations may be used in the natural languageprocessing to associate the studies using these various construct namesand other semantically similar construct names under the samestandardized construct name.

Constructs used in the study to refer to constructs that areconceptually, semantically, or ontologically related to an individualconstruct, such as subordinate constructs or superordinate constructs,may be stored as “related construct names.”

Information related to how a construct is represented, manipulated,measured and otherwise used in a study may be stored in the database andreferred to collectively as “construct instantiating information.”

The causal role for each construct in that study.

Causal roles may include independent variable, dependent variable,moderator variable, mediator variables, control variable and covariatevariable.

A document identifier that uniquely associates the information stored inthe construct database with the source document that was processed toextract that construct information.

The causal relationships may be inferred from the construct names andassociated causal roles stored in the construct database for aparticular study. For example, if a study contains four constructs(Construct A, Construct B, Construct C, and Construct D), wherebyConstruct A is indexed as an independent variable, Construct B isindexed as a moderator, Construct C is indexed as a mediator, andConstruct D is indexed as a dependent variable, one can deduce that thestudy investigated a causal relationship from Construct A to ConstructD, one from Construct A to Construct C, one from Construct C toConstruct D, and one from Construct B to the relationship betweenConstruct A and Construct D. In some embodiments, explicit informationabout the causal relationships between these constructs have beenexamined for a study may be stored in the database.

In a preferred embodiment, the construct database is populated usingNatural Language Processing and Machine Learning methods (collectivelyreferred to as “Natural Language Understanding” or “NLU”) toautomatically extract the aforementioned information from a set ofpapers, and then store them in the Construct Database. In someembodiments, some or all the information are originally specified orcorrectable using manual entry.

The construct database may also store, for each paper, metadata, such asauthor, title, publication year, journal name, other citationinformation, journal rating, number of citations to the article. Thismetadata may be referred to herein as paper-level metadata in that itdescribes information about the paper more generally. In some cases,this is distinguished from study-level metadata such as construct namesand relationships, that are associated more specifically with a studydescribed in the paper. There may be multiple studies in a paper. Insome embodiments, the metadata may include other subject matterclassification information, such as the industry or vertical to whichthe study pertains. Some metadata may be extracted from fields withinthe paper, or automatically determined based on a natural languageanalysis of the paper.

Search Overview

In a preferred embodiment, users can search based on constructs andcausal relationships between constructs for studies in the literature.

Specifically, the results are represented in the form of causal mapsthat depict the individual constructs and causal relationships that arerelevant to a user's search query. Such results are more granular thanpaper-level search results, since they visually represent study-levelsearch results within each paper, represented in terms of the constructsand relationships examined in those studies. Each paper may containmultiple studies. In some embodiments, these causal maps are aggregatedacross multiple studies within a paper. In some embodiments, thesecausal maps are aggregated across multiple papers representing a body ofliterature about a topic of interest.

Search Interface

The Search Interface allows a user to query information about particularconstructs, particular causal relationships between constructs, andparticular papers. Users can enter construct-related queries andrelationship-related queries by specifying one or more of the following:(1) one or more construct(s) of interest; (2) one or more causal rolesfor one or more of the specified construct(s), and (3) any additionalfiltering criteria based on metadata and information stored in theConstruct Database. Users can also search directly for a paper ofinterest by specifying paper-level metadata that combines one or more ofthe above mentioned construct-based criteria with one or more of themetadata criteria.

The specifications indicated above may be implemented in various ways,which include, but are not limited to, a user selecting a particularspecification from a pre-populated list (for example, for (c), thepre-populated list can include the options “independent variable,”“dependent variable,” “moderator,” “mediator,” “control variable,”“covariate”) or entering that specification in an open-ended format.

Query Engine

The Query Engine matches the input provided by the user via the SearchInterface against the information stored in the Concept Database andproduces a set of results that are subsequently displayed via the SearchResults Interface. The Query Engine performs construct identificationand result retrieval and assembly. The Query Engine can also beconfigured to accept input from components other than the SearchInterface, whereby such components can be either internal or external tothe system described in this patent application. In one embodiment, theQuery Engine can be configured to accept input from a component thatdisplays a pre-populated list of constructs, for example an ontology ofconstructs from a particular domain. A user selecting one or moreconstructs from that list would initiate a command to the Query Enginesimilar to that produced when the user specifies the names of thoseconstructs (without specifying the constructs' causal role) via theSearch Interface.

Construct Identification

If a construct of interest is specified in an open-ended format (meaningthat the user enters a particular construct name into the search field),the Query Engine initiates a semantic matching process that attempts tomatch the user-specified construct name against a correspondingconstruct name stored in the Construct Database. The construct nameidentified as matching may be an exact or a close match. For example, ifa user specifies “assortment of different products” as the constructname, the Query Engine, upon searching the Construct Database, mightfind this name to be a match for the construct name “productassortment.” Once a matching construct name has been found, the QueryEngine determines from the Construct Database the construct name that isassociated with that particular name. For example, it might determinethat “product variety” is the construct name that includes “productassortment” among its set of construct names. “Product variety” wouldhence be identified the construct most relevant to the user's initialquery.

Result Retrieval & Assembly

Once the user has submitted a query, and the construct identificationhas been successfully completed, the Query Engine retrieves from theConstruct Database all the information pertaining to the construct(s)identified as relevant to the user's query. Then, it assembles thisinformation to be used by Search Result Interface. Such informationincludes all the corresponding construct names, related construct names,instantiation information, and causal roles indexed with reference tothat construct across relevant papers, along with the paper-levelmetadata corresponding to each relevant paper.

Search Result Interface

The Search Result Interface displays the output produced by the QueryEngine. The output is shown either in a visual or textual format, or asa combination of both depending on each user's preference and the typeof task the user wants to complete. In a preferred embodiment, causalframeworks are represented in a visual format in the form of causal mapsgenerated by the Causal Mapping Engine. Papers that are predicted to berelevant to a user's query are shown in a text-based format generated bythe Textual Listings Module. One embodiment of the Causal Mapping Engineand Textual Listings Module are described below.

Search Result Interface: Causal Mapping Engine

The Causal Mapping Engine uses the output produced by the Query Engineto render graphic and interactive causal maps of all the constructs andcausal relationships that are relevant to a user's query (whereby thetotality of causal relationships that are relevant to a particularconstruct or topic form a conceptual framework for thatconstruct/topic). It represents the constructs that are part of a causalframework according to their functional role within that framework, andlinks the visually presented information back to the research papersthat were used to generate the map of the causal framework.

The logical process of mapping out a causal framework involvesinferring, based on the Query Engine output, which constructs and causalrelationships to represent in the framework (i.e., the “relevant”constructs and causal relationships), visually rendering each of thoseconstructs and relationships, and configuring the visual indicators thatallow a user to interact with each construct and relationship (by eithervisually exploring that construct/relationship in more depth, or viewinga list of the papers that are relevant to that construct/relationship).This module's visual rendering and interactivity will be discussed inmore detail in subsequent sections.

In some embodiments, causal maps can show construct relationshipsinvestigated in a single study (also referred to as a study-level causalframework, or a single-study map). In other embodiments, causal mapsshow construct relationships for multiple studies within a paper (alsoreferred to as a paper-level causal framework, or a single-paper map).In yet other embodiments, causal maps show construct relationshipsacross multiple papers (also referred to as a cross-paper causalframework, or an aggregated map).

A cross-paper causal framework is generated using a cross-paperaggregation process. It aggregates the extracted construct relationshipsfrom many research papers into one causal framework that is generallymore extensive than has been formalized or empirically tested as a wholein previous research.

When a cross-paper aggregation is generated, the Causal Mapping Enginealso generates what can be called “research volume” indicators, which,in the most basic form, could be thought of as paper counts. In apreferred embodiment, the causal mapping engine for each construct andconstruct relationship that are part of a causal framework, the enginecalculates the number of unique research papers relevant to thatconstruct and/or causal relationship. The Causal Mapping Engine countsthe papers in the Construct Database that investigates each constructincorporated in the cross-paper causal framework, and displays thosenumbers in such a way as to be associated with their respectiveconstructs in the cross-paper aggregate map. In a preferred embodiment,this association is conveyed by positioning the number on or adjacent tothe visual feature representing the corresponding construct or constructrelationship.

Instead of a paper count, other types of research volume indicators maybe used to indicate other important aspects of particular constructs andconstruct relationships. For example, a research volume indicator may becomputed as an algorithm that takes into consideration factors such as ascore of the paper's relevance to the query, the number of downloads forthat paper, the paper's publication recency, and the reputation ratingof the journal in which the paper was published.

In a preferred embodiment, a causal map is a diagrammatic representationof the constructs relevant to a user's query, represented by graphicalicons with textual descriptions of the construct, and the causalrelationships between such constructs, represented by lines or arrowsbetween such constructs.

In a preferred embodiment, an aggregated causal map includes visualand/or textual indicators showing the number of papers associated witheach construct or construct relationship. By clicking on the number orthe graphical symbol associated with a construct (e.g., an oval) or aconstruct relationship (e.g., a line), the papers associated with suchconstruct or construct relationship are presented in a textual listing.In some embodiments, multiple constructs and/or construct relationshipscan be selected simultaneously from the aggregate causal map to create atextual listing that include the papers that match the correspondingsubset of papers within the aggregated causal map.

Causal Mapping Engine: Visual Rendering Function

The Causal Mapping Engine visually represents each relevant constructonce.

Constructs are rendered as clickable graphical objects with associatedtext based description visually associated with the graphical object.These graphical objects are referred to as “construct nodes.”

Construct relationships are rendered as the corresponding causalrelationships between constructs as clickable paths (referred to as“relationship paths”),

Visual indicators for the research volume and for the list of papersthat correspond to each construct or relationship as clickable numbersor icons are displayed in the vicinity of their corresponding constructnode or construct relationship path.

The Causal Mapping Engine can render several types of maps, as definedbelow, including construct maps, model maps, and meta-analysis maps.

Construct maps assume that a construct of interest has been specified bythe user (via the Search Interface or through some other signal, likeclicking on a construct node inside a map). That construct is shown as acentral node in the map, meaning that it is causally linked viarelationship paths to all other constructs shown in the map. It may alsocontain some visual indicators that make this construct stand out fromother constructs displayed in the map. This kind of map usually showsonly relationships that involve independent and dependent variables,though other types of variables could be represented, too. Constructmaps are aggregate maps, in the sense that they aggregate causalrelationships across papers, and render research volume indicators foreach construct and relationship examined across those papers.

FIG. 4 illustrates a causal map for Construct C illustrating itsrelationships with Construct A, Construct B, Construct D, Construct E,and Construct F. Construct C is a dependent variable in therelationships between Construct C and Constructs A or B, and anindependent variable in the relationships between Construct C andConstructs D, E, or F.

The numbers shown in document icons next to each construct andrelationship exemplify the research volume for the correspondingconstruct or relationship.

FIG. 7 illustrates one embodiment of a user interface including a visualrepresentation of a construct map.

Expanded relationship maps assume that a causal relationship of interesthas been specified by the user (via the Search Interface of through someother input, like clicking on a relationship path inside a constructmap). Expanded relationship maps show the endpoints of a specifiedrelationship (i.e., the independent and dependent variable) along withall intermediary variables (i.e., mediators, moderators, and possiblycontrol variables/covariates) examined within the context of therelationship. These maps do not necessarily have a central node; insteadthe causal relationship of interest represents the central linkage thatties all the other relationships together. They may also contain somevisual indicators that make this relationship stand out from otherrelationships displayed in the map. Expanded relationship maps areaggregate maps, in the sense that they combine relationships across aset of relevant papers and also render research volume indicators.

Expanded relationship maps follow the typical representation conventionsused in social science research. As an example, the relationship betweenconstructs A and E, which involves the following constructs and causalroles: A (independent variable), B, C (moderators), D (mediator), and E(dependent variable), would be visually represented as follows:

constructs are rendered as clickable, named nodes (referred to as“construct nodes”),

the corresponding causal relationships between constructs as clickablepaths (referred to as “relationship paths”),

visual indicators for the research volume and for the list of papersthat correspond to each construct or relationship as clickable numbersor icons, which would be shown in the vicinity of their correspondingconstruct node/relationship path.

FIG. 5 shows a visual representation of a relationship map for therelationship between Construct A and Construct E, which includes thefollowing constructs and their causal roles: Construct A—independentvariable, Constructs B, C—moderators, Construct D—mediator, ConstructE—dependent variable.

The numbers shown next to each construct and relationship exemplify theresearch volume for the corresponding construct or relationship (wherebythe actual values are for illustration purposes only).

FIG. 8 illustrates one embodiment of a user interface including a visualrepresentation of a relationship map.

Model maps show the entire causal model typically tested within a studyor paper of interest (though they might also represent a conceptualframework aggregated across multiple papers). They are a hybrid betweena construct map and a relationship map, do not require a central node orcentral linkage, and they can be either aggregate maps or singlestudy/paper maps. An example of a (single-study) model map is shown inFIG. 6.

Causal Mapping Engine: Interactive Affordances

Map-to-Map Navigation. A central characteristic of the causal mapsproduced by the Casual Mapping Engine is their interactivity. Thisinteractivity is primarily aimed at allowing the user to engage witheach construct or relationship of interest directly within a causal map.

Each construct node or relationship path in a map (hereby referred to as“elements of the map”) is linked to (a) a map that conceptually relatesto that element, and (b) a list of all the papers that have examinedthat particular element. As a result, users interact with an element ofa map by visually exploring it in more depth, and/or by retrieving itslist of relevant papers.

The elements of a Causal Map are configured to be clickable, such thatclicking on any of them can result in the following:

Clicking on any construct node (or a visual indictor associated withthat node) opens up a construct map for that particular construct.

Clicking on any relationship path (or a visual indictor associated withthat path) opens up a relationship map for that relationship, whereinany moderator and/or mediator variables that have been studied forpotentially influencing that relationship are revealed.

Clicking on any research volume indicator that for a construct or arelationship (or another visual indicator that points to accessing alist of papers) shows a list of the names of all the papers that haveinvestigated that particular construct or relationship.

Within this list view, clicking on any paper name indicator (or someother visual indicator) opens up a model map of that particular paper,while clicking on the indicator for the name for a study included in apaper opens up a model map of that particular study.

A new map can open in various locations. For example, it may open withinthe current map (by “expanding” the view of the current map), in aseparate window, or in a dedicated pane adjacent to the current map.Similarly, the list of papers relevant to a particular construct orrelationship may be shown in various locations such as a separate windowor a dedicated pane adjacent to the current map.

An example of a construct map for the construct “product variety” isshown in FIG. 4.

A user interested in the relationship between product variety and choicecan click on the path linking the two constructs. Doing so expands theview of that relationship and shows which additional constructs (whichin this case include moderating and mediating variables) have beentested in the context of that relationship (whereby directionality andresearch volume for a mediating or moderating relationship arerepresented the same way as in the construct map in FIG. 4).

Once the user has identified a mediating or moderating relationship shewishes to learn more about, she can click on the research volumeindicator for the corresponding path (or another visual indictorassociated with that path) to access a text-based view of the searchresults for that relationship, which is generated by the TextualListings Module (described further below).

Description for the Visual Mapping System

The aggregated visual maps can be generated as detailed below.

Construct Map Generation

Whenever a user specifies only one construct of interest, the user isshown a Construct map. In more detail, the process of generating aConstruct map can include the following steps:

In Step 1, the system identifies in the Construct Database all thestudies in which the construct of interest has been examined in thecausal role of independent variable.

In the second step, for each study identified in Step 1, the systemidentifies from the Construct Database (1) the paper that is associatedwith that particular study and (2) all the constructs that have beenexamined as dependent variables in that study (hereby called“outcomes”), and creates an output such as a table—aggregated acrossstudies—that directly associates each outcome with the paper itcorresponds to.

An example of such an output is shown below: Study 1-1: Paper 1, OutcomeA; Study 1-2: Paper 1, Outcome A; Study 1-3: Paper 1, Outcome B; Study2-1: Paper 2, Outcome B; Study 2-2: Paper 2, Outcome C; Study 3-1: Paper3, Outcome A; Study 3-2: Paper 3, Outcome C

In Step 3, based on the information generated in Step 2, the systemproduces a list of all the unique outcomes identified in Step 2, wherebyany outcome that appears more than once in Step 2 is listed only once.For example, based on the table above, the list of outcomes wouldinclude outcomes A, B, and C.

In Step 4, for each outcome identified in Step 3, the system creates alist and a total count of all the unique papers that a particularoutcome was associated with, whereby a paper that is associated with thesame outcome more than once is listed and counted only once. Forexample, based on the table above, outcome A would be associated with alist that includes papers 1 and 3 (for a total count of 2 papers).

Steps 5-8 are repetitions of Steps 1-4, with the distinction that Step 1identifies all the studies in which the construct of interest has beenexamined in the causal role of dependent variable, which leads to thegeneration of a list of predictors in Step 7 and a list of papers (perpredictor) in Step 8. Steps 1-4 and 5-8 can be performed eithersequentially or in parallel.

In Step 9, the system then displays the retrieved information in theform of a visual map, in which the construct of interest is representedas the central node in the map. On the right-hand side of the constructof interest are all the constructs listed as outcomes in Step 3, and onthe left-hand side of the construct of interest are all the constructslisted as predictors in Step 7. The construct of interest and theoutcomes are connected via visual, clickable paths (which can be pointedtowards the outcomes, so as to better reflect the direction of thecausal relationship), whereby each path is accompanied by an icon thatprovides access to the list of papers generated in Step 4 and canpotentially also show the count of those papers. The construct ofinterest and the predictors are also connected via visual, clickablepaths (which can be pointed towards the construct of interest, so as tobetter reflect the direction of the causal relationship), whereby eachpath is accompanied by an icon that provides access to the list ofpapers generated in Step 8 and can potentially also show the count ofthose papers.

Whenever a user specifies only one construct of interest, along with thecausal role for that construct (i.e., either independent or dependentvariable), the information shown on the construct map is reduced to thefollowing: if the construct of interest is specified as an independentvariable, then the predictors and the visual paths connecting eachpredictor to the construct of interest are not shown. If the constructof interest is specified as a dependent variable, then the outcomes andthe clickable paths connecting the construct of interest to each outcomeare not shown.

Relationship Map Generation

Whenever a user specifies two constructs of interest, along with thecausal role for each construct (i.e., independent variable for oneconstruct, and dependent variable for another), the user is shown aRelationship map. A Relationship map is a scaled-down version of aConstruct map, which shows only the two constructs specified by theuser, with the construct specified as independent variable shown as apredictor, the construct specified as the dependent variable shown as anoutcome, and a visual path (along with the corresponding icon)connecting the predictor to the outcome. A Relationship map can be shownin a condensed format, which includes only the endpoints of arelationship (predictor and outcome), or in an expanded format, asdetailed in the previous section.

The Causal Mapping Interface also includes specific features such as aMap Content Filtering feature and a Map Display Adjustment feature,which are detailed below.

Map Content Filtering. Once the content of a causal map has beengenerated, a user can control the overall content of that map via a MapContent Filtering feature. The Filtering feature uses the outputgenerated by the Query Engine to display various categories ofinformation about each construct (or causal relationship) shown in acausal map, and allow the user to narrow down the content of that map bymeans of selecting particular values for each category of informationfor that construct (or relationship). For a particular construct, theFiltering feature could, for example, list all the values for thatconstruct's names, related construct names, and/or instantiatinginformation (whereby those values are taken from each of the relevantstudies/papers that were returned by the Query Engine and subsequentlyused by the Causal Mapping Engine for generating the current causalmap). Per each category of information, the user can select (via acheckbox or some other selection aid) what values she wants to seeincluded for that category of information. For example, for theconstruct “product variety” and the information category “constructnames,” the user may see four values, “number of attributes,” “number ofbrands,” “product line length,” and “product line depth,” and she mayde-select the last two values. Upon the user making her selection, arequest is sent to the Causal Mapping Engine to automatically removefrom the generated causal map all the papers that do not contain atleast one study matching the user's specified criterion. Thiselimination can (though does not need to) result in a visually modifiedcausal map containing a lower number of construct nodes, correspondingunique causal relationships, and/or a lower research volume for one ormore causal relationships. The Filtering Feature may also use the outputgenerated by the Query Engine to display filtering criteria based onpaper-level metadata, such as publication year, journal rating, numberof citations, etc. The Filtering Feature can be shown in one or moreplaces in the Causal Mapping Interface (for example directly adjacent toa causal map). Some of its functionality can also be combined with thatof the Search Interface.

Map Display Adjustment. Upon viewing a causal map result, particularly aresult with many discrete constructs, the user can manage its size,location, or overall display in one or more ways. For example, the usercan zoom in or out of viewing the map by clicking on icons to the sideof it (e.g., magnifying glass icons). Alternatively, the user can dragthe map, by clicking, holding, and moving any portion of it, to whateverposition she may choose. By default, the Causal Mapping Engine couldrender all maps, however large and/or complex, to fit-centered-withinits mapping pane, so that the user can immediately see the causalframework of interest. However, if the map rendered by the CausalMapping Engine is too large and/or complex for the user to immediatelyunderstand (or too large for the user's comfort), the user can choose toadjust their viewing of the map via the Map Display Adjustment feature.

An alternative to the Causal Mapping Engine described so far is anAssociative Mapping Engine, which visually represents constructs thatco-occur across papers, or which are conceptually related, but whichdoes not visually depict any types of relationships between theseconstructs. The Causal and Associative Mapping Engines can operate inparallel, or can be combined into one engine. The user can choose, forexample via the Map Display Adjustment Feature, whether he wants to seea causal mapping or an associative mapping view of the results, and canpotentially switch between the two types of views.

Construct/Relationship Subscription Management. By clicking on aspecific relationship path or construct node (or on particular visualindicators associated with a path or node), users will be able tosubscribe to receiving electronic notifications (e.g. emailnotifications) of any papers or studies that have been newly added tothe system's Construct Database and that match the construct orrelationship clicked on. The Construct/Relationship SubscriptionManagement feature is also accessible from the Textual Listings Module.

Textual Listings Module

This module is invoked whenever a user chooses to see the output of asearch query in a text-based format. It displays the results of theQuery Engine as a list of constructs and/or relationships that arerelevant to the user's query. The results can be ranked according to avariety of factors, such as the corresponding research volume perconstruct/relationship.

This module is also invoked when a user wants to see a list of papersthat relate to a particular construct or relationship within a causalmap. In that case, the module produces textual listing of only thosepapers that deal with the causal relationship of interest, whereby thepapers can be grouped according to such constructs and relationships.For each paper, the module can show, for example, the name of the paper,the name of the papers' authors, the publication year, or any otherinformation based on the metadata values available for that paper. Themodule can also show and/or embed links to one or more locations fromwhich those papers can be retrieved. Such locations may include, but arenot limited to, the paper's DOI location (i.e., the universal andpermanent internet address of a published paper), an external databaseor digital library that contains the paper, a location on the internalserver where the paper is stored, etc. The papers may be orderedaccording to a variety of criteria, such as publication year, number ofcitations, journal ranking, etc.

Error Reporting. Upon viewing the textual listings of papers on aparticular construct and/or relationship, users can submit to the systema report about any errors they discern in the ways in which the systemhas apparently associated those papers to the construct(s) orrelationship(s) of interest. They can also provide other error reportssuch as suggesting papers that are relevant to a particular construct orrelationship and which do not appear in the Search Result Interface.Such error reports can then be transmitted to the Query Engine module,or to the Construct Database, either in their original form, or afterbeing modified by a third party, and would serve for use in laterimprovements to the indexing/classification of papers, constructs,and/or relationships.

In more detail, the workflow for a user query comprises the followingprocesses illustrated in FIG. 9:

The user starts off by indicating via the Search Interface whether sheis interested in retrieving information about a construct, arelationship between constructs, or specific papers of interest (thelast of which is based on paper-level metadata values). This indicationmay be done explicitly (whereby the user selects a particular option) orimplicitly (whereby the user fills in certain entry fields, but notothers).

If the user is interested in a particular construct, she will beprompted to enter a name for that construct into the Search Interface.She can further specify whether she wants a particular causal role(i.e., independent variable, dependent variable, mediator etc.) assignedto that construct.

The Query Engine automatically compares the user-specified name againstthe construct names stored in the Construct Database, and determineswhether a matching construct name can be found.

If no matching construct name can be found, a message is returned to theuser. This message can, for example, indicate the absence of a matchingconstruct name and recommend a list of alternative construct names forthe user to choose from.

If a matching construct name can be found, then the system identifies inthe Construct Database the construct name that is associated with thatparticular name.

If the user is interested in a particular relationship betweenconstructs, she will be prompted to enter a name for each construct intothe Search Interface. She can further specify whether she wants aparticular causal role assigned to each construct.

The system will perform Steps i-iii until the construct name associatedwith each user-specified name has been identified.

If the user is interested in a particular paper, she will be prompted toenter identification information about that paper (such as authors,title, etc.) into the Search Interface. The Query Engine will attempt toidentify that paper in the Construct Database.

If no matching paper can be found, a message is returned to the user.This message can, for example, indicate the absence of a matching paperand recommend a list of alternative paper for the user to choose from.

Once the relevant construct name(s) or the relevant paper have beensuccessfully identified, the user will select whether she wants to see atext-based or a visual representation of the query results.

If the user selects a text-based representation of the results, theTextual Listings Module will generate a list of relationships (describedin terms of the constructs included in each relationship and thecorresponding causal role for each construct) and/or papers that arerelevant to the user's query.

If the user selects a visual representation of the results, she will seea causal map generated by the Causal Mapping Engine as follows:

If the search query specified a construct of interest, the user willfirst see a construct map. She can then choose to click on anyrelationship of interest within the construct map to view a relationshipmap. She can alternatively choose to click on any research volumeindicator (or another visual indicator) in the construct map to view alist of relevant papers, as generated by the Textual Listings Module.She can then choose to review the textual information available aboutany papers of interest (including following the links that direct her toeach paper's location, which can be either inside or outside the presentsystem), or click on any paper name (or some other visual indicatorassociated with that paper) to see a model map for that paper (or for astudy in that paper).

If the search query specified a relationship of interest, the user willsee a relationship map. She can additionally choose to click on anypaper count indicator (or another visual indicator) in the relationshipmap to view a list of relevant papers, as generated by the TextualListings Module. She can then choose to review the textual informationavailable about any papers of interest (including following the linksthat direct her to each paper's location) or click on any paper name (orsome other visual indicator associated with that paper) to see a modelmap for that paper (or for a study in that paper).

If the search query specified a paper of interest, the user will see amodel map for that paper, along with any textual information availableabout the paper. She can then choose to click on any individual study tosee a model map for that study.

FIG. 11 is a diagrammatic representation of an embodiment of a machine900, within which a set of instructions for causing the machine toperform one or more of the methodologies discussed herein may beexecuted. The machine may be connected (e.g., networked) to othermachines. In a networked deployment, the machine may operate in thecapacity of a server or a client machine in a client-server networkenvironment, or as a peer machine in a peer-to-peer (or distributed)network environment. In one embodiment, the machine communicates with aserver to facilitate operations of the server and/or to access theoperation of the server. In some embodiments, the machine may act as aserver for some functions and a client for other functions.

In some embodiments, the machine 900 runs the application 100 orapplication 200. In other embodiments, the machine 900 is the search,visualization and navigation operations according to an embodiment asdescribed herein or a component of such systems, such as one or moremodules or units described herein. In other embodiments, the machine 900is the Construct Database according to an embodiment as describedherein.

The machine 900 includes a processor 960 (e.g., a central processingunit (CPU), a graphics processing unit (GPU) or both), a main memory 970and a nonvolatile memory 980, which communicate with each other via abus 902. In some embodiments, the machine 900 may be a cluster ofcomputers or comprise multiple processors or multiple processor cores.In one embodiment, the machine 900 also includes a video display 910, analphanumeric input device 920 (e.g., a keyboard), a cursor controldevice 930 (e.g., a mouse), a drive unit 940 (e.g., solid state drive(SSD), hard disk drive, Digital Versatile Disk (DVD) drive, or flashdrive), a signal generation device 950 (e.g., a speaker) and a networkinterface device 990.

In some embodiments, the video display 910 includes a touch-sensitivescreen for user input. In some embodiments, the touch-sensitive screenis used instead of a keyboard and mouse. The drive unit 940 includes amachine readable medium 942 on which is stored one or more sets ofinstructions 944 (e.g. software) embodying any one or more of themethods or functions of the inventive subject matter.

The instructions 944 may also reside, completely or partially, onmachine-readable media within the main memory 940 and withinmachine-readable media within the processor 960 during execution thereofby the machine 900. The instructions 944 may also be transmitted orreceived over a network 995 via the network interface device 990. Insome embodiments, the machine-readable medium 942 also includes adatabase 944 including some of the received information.

While the machine-readable medium 942 is shown in an exemplaryembodiment to be a single medium, the term “machine-readable medium”should be taken to include a single medium or multiple media (e.g., acentralized or distributed database, and/or associated caches andservers) that store the one or more sets of instructions and/or data.The term “machine readable medium” shall also be taken to include anynon-transitory medium that is capable of storing, encoding or carrying aset of instructions for execution by the machine and that cause themachine to perform any one or more of the methods or functions of theinventive subject matter. The term “machine-readable medium” shallaccordingly be taken to include, but not be limited to, solid-statememories, optical and magnetic media, and other non-transitory tangiblemedia.

In general, the methods executed to implement the embodiments of thedisclosure, may be implemented as part of an operating system or aspecific application, component, program, object, module or sequence ofinstructions referred to as “programs.” For example, one or moreprograms may be used to execute specific processes according to theinventive subject matter. The programs typically comprise one or moreinstructions set at various times in various memory and storage devicesin the machine, and that, when read and executed by one or moreprocessors, cause the machine to perform operations to execute methods,functions and other elements of the inventive subject matter.

Moreover, while embodiments have been described in the context ofmachines, those skilled in the art will appreciate that the variousembodiments are capable of being distributed as a program product in avariety of forms, and that the disclosure applies equally regardless ofthe particular type of machine or computer-readable media used toactually effect the distribution. Examples of machine-readable mediainclude, but are not limited to, recordable type media such as volatileand non-volatile memory devices, flash memory devices, floppy and otherremovable disks, hard disk drives, and optical disks such as CompactDisk Read-Only Memory (CD-ROMS) and Digital Versatile Disks (DVDs),among others.

1. A computer-implemented method for querying a database for a specifiedconstruct and returning query results in the form of a construct map,said construct map being an aggregated visual map, said aggregatedvisual map having the specified construct visually represented as acentral node.
 2. The method of claim 1 wherein the specified constructhas a causal role of an independent variable and said construct map hasthe visually represented constructs other than the specified constructdisplayed to the right of the specified construct.
 3. The method ofclaim 1 wherein the specified construct has a causal role of a dependentvariable and said construct map has the visually represented constructsother than the specified construct displayed to the left of thespecified construct.
 4. The method of claim 1 wherein said aggregatedvisual map is in the form of a meta-analysis map, said meta-analysis mapdisplaying, for at least one relationship included in the meta-analysismap, one or more textual or graphic elements indicating a sign or astrength for said at least one relationship.
 5. A computer-implementedmethod for querying a database for a specified causal relationship andreturning query results in the form of an expanded relationship map,said expanded relationship map being an aggregated visual map, saidaggregated visual map having the specified causal relationship visuallyrepresented as a central linkage.
 6. The method of claim 5 wherein saidaggregated visual map is in the form of a meta-analysis map, saidmeta-analysis map displaying, for at least one relationship included inthe meta-analysis map, one or more textual or graphic elementsindicating a sign or a strength for said at least one relationship.
 7. Acomputer-implemented method for querying a database for a paper andreturning query results in the form of a model map, said model map beingan aggregated visual map or a single study map, depending on the numberof studies contained in the paper.
 8. A computer-implemented system forconstruct- and study-based document indexing comprising: a studyidentification unit configured to identify one or more studies indocuments; a construct and causal role identification unit configured toidentify two or more constructs that were manipulated or measured ineach study and to determine at least one causal role for each identifiedconstruct within each study, the causal role comprising at least one ofan independent variable, a dependent variable, a mediator variable or acovariate; a database configured to store in a computer-readable mediumthe identified two or more constructs with the determined causal role[s]for each of the identified two or more constructs within each study, afirst identifier for each study associated with a construct and a secondidentifier for the document associated with each study.